• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    System and procedure for detecting the occupation status of the parking spaces (PAs) defined. The system comprises: video camera (1.1), artificial vision algorithm processing module (1.2), rangefinder orientation control algorithm (1.3), pan-tilt steerable platform (1.4), laser rangefinder, pole height 1.5 m and reflection prism to obtain the distances in the calibration stage (1.5), differential measurement algorithm of the distances (1.6), fusion module of the data coming from the camera, the rangefinder (1.7) and communication module (1.8). The dividing line leaves the hardware elements of the system to the left, while the information processing algorithms are to the right of that line. The different elements are linked and listed as it becomes necessary to use them in the system. In the installation of the system you need to perform a calibration procedure of the rangefinder + camera set. (Machine-translation by Google Translate, not legally binding)
    公开号:ES2680993A1
    申请号:ES201700190
    申请日:2017-03-10
    公开日:2018-09-11
    发明作者:Alfredo Gardel Vicente；Ignacio Bravo Muñoz；José Luis LÁZARO GALILEA；Felipe Espinosa Zapata
    申请人:Universidad de Alcala de Henares UAH；
    IPC主号:
    专利说明:

    SYSTEM AND PROCEDURE FOR THE DETECTION OF THE OCCUPATIONIN PARKING
    1 SECTOR OF THE TECHNIQUE This invention falls within the area of the Electronic, Computer and Telecommunications (ICT) technique.
    2 STATE OF THE TECHNIQUE
    2.1 Introduction The parking of vehicles in urban and densely populated areas represents a major problem in relation to mobility and transport. The existence of vehicles in search of parking means a great loss of time and money for citizens, and an additional energy expenditure that leads to greater environmental pollution.
    The different public administrations and concessionary companies would like to be able to inform the driver of the location of a free space for parking. In this patent we focus on the detection of the occupation status (free / occupied) of a series of parking spaces (PAs) within the area supervised by a sensor node (camera-rangefinder) leaving open the way in which said information is transmits / communicates to the driver (e.g. through a mobile phone application).
    Systems such as the one presented in this patent are intended to provide information about the availability of free PAs in an area under supervision. Next, the state of the art on the different existing technologies is briefly reviewed to know if a PA is or is not occupied by a vehicle distinguishing whether a sensor detects the occupation of a place or several places.
    2.1.1 Systems with an occupancy sensor in each PA
    In this strategy an occupancy sensor is installed for each PA. In [Chinrungrueng, J., Sunantachaikul, U. & Triamlumlerd, S., 2007. Smart Parking: An Application of Optical Wireless Sensor Network. https: /ldx.doi.org/10.1109/SAINT-W.2007.981 an optical sensor is used in each PA to inform about its occupation. Other types of ultrasonic-based sensors can also be used [Burgstahler D, Knapp F, Zoller S. "Where
    is that car parked A wireless sensor network-based approach to detect car positions ". 9th
    IEEE Workshop on Practicallssues in Building Sensor Network Applications; 20 14 Sept. p.
    1-11. https: lldx.doi.org/10.1109/LCNW.20 14.6927697] or based on magnetic sensors
    [Zhang, Z., Tao, M. & Yuan, H., 2015. A Parking Occupancy Oetection Algorithm Based on
    5 AMR 5 sensor. 15 (2), pp. 1261-1269. https://dx.doi.org/10.1109/J5EN.2014.2362122).
    There are several commercial systems that incorporate a sensor for each PA as per
    MeshNetics example, but its use has been relatively short since the
    maintenance of the nodes has a high cost, they need to implement a large
    infrastructure and are very invasive systems.
    10
    2.1.2 Systems with occupancy sensor of several PAs
    This section considers those systems that obtain the occupation status of
    each of the PAs in an area under supervision (without covering all parking), for example
    through artificial vision [Ball, Jay H. "Oetermining the availability of parking spaces." US
    lS Patent 62852972001 https: llwww.google.es/patents/US6285297]. These systems network ucen
    clearly the costs of installation and maintenance since it has a relation N to
    1, where N is the number of places supervised by a single detection node. By cons,
    the reliability of the measurements made by this type of remote systems is worse than the
    reliability obtained by sensor-based systems by monitoring a single PA.
    twenty
    The next section focuses on the state of the art in the revision of vision algorithms
    artificial used in obtaining the occupancy of PAso
    2.2 Remote detection based on artificial vision algorithms
    2S Then, once reviewed the systems that make use of different techniques
    The study of the state of the art is focused to know the occupation of the PAs
    in systems based on artificial vision1.
    2.2.1 Detection of soil and PA marks
    30 One of the most used strategies in artificial vision is to obtain the occupation status
    of a PA based on the detection of soil / pavement. In the work of [Wu, a.w.a. et aL,
    2007 Robust Parking Space Detection Considering Inter-Space Correlation. Multimedia
    and Expo, 2007 IEEE Inl. Conference https: /Idx.doi.org/10.11 09 / ICME.2007,4284736) se
    makes use of machine learning methods to classify each of the PAs as
    busy or free These techniques require prior training that takes into account the negative / positive detection of vehicles in the PAs. It is also possible to use Bayesian classifying algorithms such as the work of [Liu, J., Mohandes, M. & Deriche, M., "A multi-classifier image based vacant parking detection system N. ln 2013 IEEE 20th Int. Conference on Electronics, Circuits, and Systems (ICECS). IEEE, pp. 933-936. Https: //dx.doi .org / 10.1109 / 1CECS.2013.6815565J The alteration of the appearance of the ground due to changes in lighting, shadows, occlusions due to the passage of vehicles, etc. and the appearance of vehicles that can partially occupy the square, reduce The effectiveness of these machine learning methods greatly.
    Other works such as the [Zhang, G., Feng, Y. & Wang, B. uParked vehicle detection based on edge detectionN US Patent 8923565, 2014
    •
    https://www.qooqle.es/patents/US8923565] try to exploit the information of the marks of places on the parking floor. This type of solutions is dependent on the good condition of the markings on the pavement, a correct parking by the users and a high image resolution to correctly segment said marks.
    2.2.2 Detection and tracking of vehicles in the parking lot
    In other works, in addition to the above, they propose to detect and track moving vehicles through the parking lot. In [Faro, A, Giordano, D. & Spampinato, C. "Adaptive background modeling integrated with luminosity sensors and occlusion processing for reliable vehicle detection. IEEE Trans. On Intelligent Transportation Systems, 12 (4), pp. 1398--1412 2011. https://dx.doi.org/10.1109/T1TS.201 1.2159266J proposes a vehicle detection and tracking system based on the subtraction of the background image with an adaptive Poison model. When parking a vehicle, it may improve the obtaining of a PA occupation or not, however, the authors themselves warn of system errors due to brightness in the image or poor lighting, in order to be able to track the vehicles in the areas annexed to the PAs should extend the working area of the cameras and make use of other algorithms such as those based on optical flow [Park, S .; Kim, K .; Park, K. "Vehicle-monitoring device and method using optical flow ".
    U.S. Patent 20060140447, 2005. https://www.google.com/patents/US20060140447). From this information, moving objects are detected and classified as vehicles
    or not. There are multiple authors who make use of these algorithms, citing for example the work of [Blumer, K. et aL, 2012. "Cost-effective single-camera multi-car parking
    monitoring and vacancy detection towards real-world parking statistics and real-time reporting ". In Neural Informalion Processing. pp. 506-515. https: /Idx.doi.org/10.1007/9783-642-34500-5 60] in the which is an analysis of dynamic objects based on the information given by the optical flow. 2.2.3 Improvements to the artificial vision system
    Additionally, there are other works that are commented on later and that can improve the obtaining of results of the previous methods by including more information in the decision making about the state of occupation of the PAs, such as:
    • the 3D model of the car park by having a fixed perspective between the supervision chamber and the car park floor and vehicles moving through it,
    • detection and recognition of vehicle license plates, provided that the resolution and perspective allows it.
    2.2.3.1 3D parking model Several authors propose to make a parking model 30. Thus, in [Huang, C.-C., Tai, Y.-S. & Wang, S.-J, 2013. "Vacanl Parking Space Deleclion Sased on Plane-Sased Bayesian Hierarchical Framework H. IEEE Transactions on Circuits and Systems for Video Technology, 23 (9), pp. 1598-1610. Https: / Idx. doi.org/10.1109rrCSVT.2013.2254961J a detection algorithm is made that, using a 3D model of the car park, improves the detection results by taking into account occlusions between the different objects and vehicles moving through the car park.
    In the patent [Delibaltov, D .; Wu, W .; Loce, R. & Bernal, E. "Method of determining parking lot occupancy from digital camera imagesH US Patent 9129524, 2015,
    •
    https://www.google.com/Datents/US9129524] an estimate is made of the 3D volume occupied by each parking space to take into account the perspective of the captured image.
    2.2.3.2 License plate identification Some systems such as the one presented in [Prieto, PR, "System for estimating the location of vehicles in parking 10tsH. 2009. US Patent 7619542. http://www.google.es/patents/US7619542] do Use of cameras to detect the registration of the vehicle at certain control points in the parking area, being able to indicate to the central parking system the areas for which said vehicle is located and to be able to limit the number of vehicles that are parked in a certain area. Also in the patent [Nerayoff, S. & Wong, T. ~ Controlling use of parkíng spaces using cameras and smart sensors ". 2015. US Patent US8982214-B2 http://www.google.es/patents/US8982214J the vehicle license plates are detected to report if a PA has been occupied.
    2.2.4 Selection of vision algorithms
    To select the best viewing algorithm, a database of videos of real car parks can be used, such as that provided publicly in [Almeida, et al., 2015. WPKLot -A robust dataset for parking lot classification. Expert Systems with Applications, 42 (11), pp. 3937-4949. httds: lldx.doLorg / 1 0.1 016 / i.eswa.2015.02.009]. In this dataset there are images captured in 2 car parks from 3 different cameras contemplating complex situations: cloudy, sunny days with variable shadows, rainy, etc. Datasef has been used and tested with different algorithms at work [Almeida, P. et aL, 2013. Parking Space Detection Using Textural Descriptors. In 2013 IEEE International Conference on Systems, Man, and Cybernetics. IEEE, pp. 36033608. https: /ldoi.org/10.1 109 / SMC.2013.614]. The results confirm the difficulty of obtaining correct results about the state of the PAs from image processing only, especially if there are shadows on sunny days, wind that produces movements / wobbles of elements of the scene and brightness / poor detection of objects in low lighting conditions.
    It should be noted that the systems based on artificial vision have errors in the results of detecting the status of the occupation for multiple reasons such as the brightness / shadows produced by vehicles moving through the parking lot, the partial occlusions of the seats that produce vehicles from neighboring plazas, etc.
    2.3 Remote sensing based on a laser rangefinder The introduction of an adjustable rangefinder in the present invention makes it necessary to review the state of the art in terms of currently existing rangefinder modules and in particular those intended for operation in outdoor environments. The current technology allows to have laser rangefinders for outdoors that without handling high powers that can be harmful to people, allow to obtain a measure of the distance between the system and the surface of the first object that reflects the beam of the rangefinder, reaching 300m in conditions of good reflectivity. In the case of monitoring parking areas, the distances that are considered optimal for camera-based monitoring systems are 30-40m maximum between the location of the camera and the PAs of the vehicles. In accordance with the data sheet of the manufacturers of the rangefinder devices for the distances handled with a reflectivity of only 1.0% on the surface, a correct distance measurement is obtained [Pfeifer N. and Briese C. ~ Laser scanning -principies and applications ". GeoSiberia 2007. https: /Idx.doLorg/10.3997/2214-4609.201403279).
    Different authors have used it to obtain the occupation of PAs, for example in the patent [Wang, Y .; Cummins, D .; Darst, M. & Pennington, G. "Methods, systems and processor-readable media for parking occupancy detection utilizing laser scanning ~. 2015. US Palenl US20150116134-A 1. https: f / www.google.com/palenls / US20150116134) where an adjustable laser rangefinder system is used by means of horizontal and vertical (pan-tilt) rotation platform as in the present invention.At each profile swept by the sensor a series of distances is obtained that allow to know whether or not there is a vehicle in a PA In this way the system provides an occupancy value based on the interpretation of said distance profile.The handicap is that the entire parking area must be swept constantly.
    Another system also based on the detection of distances is the one presented in the patent [Mimeault, Y. "Parking management system and method using lighting system-. 2014. US8723689-B2. Https://WNW.google.es/patentsfUS87236891 where It is proposed to locate a rangefinder with multiple laser beams and that retrieves the distance information at different points of a series of PAso. The difference in distances depending on whether or not a vehicle exists in the PA determines the occupancy status of each PA.
    However, systems based solely on the measurement of distances may introduce errors due to the capture of surfaces that do not have to be a vehicle but that could have a similar pattern due to occlusions of multiple vehicles in the parking lot or other situations. This fact limits the use of this type of systems in areas with few PAs supervised from the same sensor node.
    2.4 Conclusions and motivation of the present invention From the analysis of previous works related to this patent, and commented in this section, the conclusion is drawn that there is currently no reliable detection system for the occupation of PAs, so that the merger and adequate treatment of the information from a video camera and an adjustable rangefinder as proposed in this invention, represents an advance in obtaining the occupation of the PAso Systems as proposed in the present invention provide great added value to the service of surface car parks, so in addition to city councils and other public entities, private parking managers / administrators may also be interested in being able to incorporate information and guidance systems to these facilities to help the user park their vehicle. In this case, the specific objective pursued with the system of this patent is to know the occupancy status of each parking space with a reduced installation cost per space.
    3 DESCRIPTION OF THE INVENTION The present invention proposes a device that obtains the state of occupation of each of the PAs within the area supervised by a sensor node that is composed of the following elements: video camera, laser rangefinder on pan platform. filt and processing system. The objective for which it has been developed is the reliable detection of the occupation status of the PAs of vehicles.
    The joint operation of the different component elements and the fusion of data obtained by both sensors in the processing system provides values of occupancy detection of correct seats even in low lighting conditions, brightness, occlusions, shadows, flashes in the lighting, night operation, etc.
    The system configuration is open, being able to change the operation of the set by modifying the different parameters that control the operation of the vision algorithm. The merging of the information provided by the vision algorithm regarding the occupation or not of a place is combined with the information provided by the rangefinder properly oriented towards classified places with a possible change in its status.
    The number of supervised places varies depending on the angle of inclination of the camera, the height of placement of the camera and the layout / size of the defined PAs. The system does not need a fixed visual demarcation of the places on the ground, being able to vary its structure, position and dimensions of the PAso
    The remote sensing system is composed of a vision camera with a certain field of vision and an adjustable laser rangefinder with a pan-tifl motion control that covers the points of interest of the PAs within the camera's field of vision. Making joint use of the laser rangefinder for outdoors improves the detection in the occupancy values of the seats in a remarkable way, especially in low lighting conditions since a second check of the results given by the visual detection module is available, exceeding in performance to other systems that are only based on vision or range finder.
    In the present invention, the laser rangefinder is complemented by an adjustable 2-axis, pan-tilt movement platform, which allows it to be directed to any parking place and point under supervision, extracting information about the distance to said point. The novelty of the patent lies precisely in the merging of the information obtained from both systems: image of the square, recognition of the space (ground) that corresponds to each PA, motion detection of a car in an area in the sequence of images that it can mean that it occupies or leaves a PA near that area free, and the measurement of distances to the PAs of interest in the affected area so that there are more reliable PA occupancy results, allowing its use in real parking systems .
    The main features of the present invention are the following:
    one. The proposed architecture of the system consists of a camera plus rangefinder on a pan-tilt movement platform, as a single set of remote sensing for the detection of the occupancy status of PAso
    2. Performing a geometric calibration (in the installation) of the rangefinder-camera assembly to be able to conveniently orient the laser rangefinder to measure the distance to points of the image that are considered necessary to ensure the correct detection of the occupation of a place.
    3. Data fusion algorithm and its application to the detection of PA occupancy consisting of the following phases: recognition and possible identification of the land corresponding to each place; motion detection of entry or exit of a vehicle from a PA from the sequence of images given by a camera; preclassification of a possible change in occupancy in a place monitored from the vision algorithm; orientation of the telemeter towards those places and distance measurement; data merger and final decision to change the state of the square taking into account the different volumes and occlusions produced by vehicles parked in adjacent squares to reliably generate the occupancy status of each PA
    DESCRIPTION OF THE FIGURES To complete the description of the invention and in order to help a better understanding of the characteristics of the system, a set of schematic figures is attached as an integral part thereof, in which, For illustrative purposes and not limitation, the following has been represented.
    Figure 1 shows a diagram with the different elements that make up the proposed system as they are used: video camera (1.1), module for processing artificial vision algorithms (1 .2), Algorithm orientation control algorithm (1.3), pan-tilt adjustable platform (1.4), laser rangefinder, 1.5m height pole and reflection prism to obtain distances in the calibration stage (1 .5), measurement algorithm distance differential (1.6), data fusion module from the camera, rangefinder (1.7) and communication module (1.8). The dividing line leaves the hardware elements of the system to the left, while the information processing algorithms are to the right of that line. The different elements are chained and listed as their use in the system becomes necessary.
    Figure 2 shows the scheme of the different tasks and procedures to perform the installation and calibration of the system: Definition of PAs and measuring points of the rangefinder for each of the places (2.1); Geometric calibration between chamber and floor of the PAs (2.2); Calibration of the laser rangefinder orientation system obtaining the pan-tilt control angular values (2.3); Calibration of the Camera-Range set by storing a correlation table between the pan-tiIt motion controls and the x-y coordinates of the image pixel for the points of interest of the parking lot PAs (2.4); Determination of delimited spatial volumes of each of the PAs and occlusions that it generates in adjacent squares (2.5).
    A schematic drawing of the measurement of distances from an adjustable rangefinder with two degrees of pan-tilt freedom is shown in Figure 3. In the figure, the horizontal pan angle and the vertical tilt angle are marked with rotation arrows. The figure shows a simple example of surface parking divided into 6 seats and with 2 vehicles of a different size parked. In the diagram, the possible measures of distances from the conveniently oriented rangefinder to a reflection prism are marked with dashed lines. In the P1 position, the distance measurements to the center of said PA obtained in the calibration phase have been marked. The center of the square is characterized by two points, one on the ground, providing the distance Osc; and another at a certain height achieved with an auxiliary pole, providing the distance Dpc. Note: in this document a height of 1.5 m is considered as the average height of a vehicle and therefore will be the height of the post, but you can choose a different one.
    In the P2 square, the measured distances to the points of interest have been marked, obtaining the current OPA and OSA distance values for that square. As you can see, in the case that there are no occlusions in a free space, the OSA value must be similar to the calibrated Dsc value. while the OPA value will be different from the calibrated Opc value (marked with a gray line in the cut with the broken line of 1.5 m height).
    In the P3 square there is a parked vehicle and the measured distances to the points of interest have been marked, obtaining the current OPA and OSA distance values for said square. As you can see, in the case of a parked vehicle, the OPA value must be similar to the calibrated Opc value. while the OSA value will be different from the calibrated Osc value. The case of the P5 square shows the case of an older vehicle
    dimensions and displaced with respect to the location centered on the square.
    The P6 square has ground occlusion so that the OSA distance measurement from the rangefinder cannot be performed. Only the OPA measure to compare with the calibrated Opc distance is available. As shown in the figure, the ground point of a PA on which to obtain the distance measurement can easily be occluded by other adjacent vehicles. This motivates having included a point at a certain height (1.5 m) so that the probability that the distance measurement obtained from the rangefinder has occlusions is reduced,
    The difference in distances measured at points of a PA with a vehicle or without a vehicle provides additional information to be combined with the results of the artificial vision module and the parking status.
    10 Figure 4 shows the procedure to preclassify a PA as occupied. The procedure is executed if the movement of a possible vehicle (from the camera images) that has stopped in the area of the square is detected. Thus, considering a place Pi, if near it the movement of a vehicle with a value T> UT (exceeding a certain threshold UT) is stopped and the recognition value of its floor RI is less than a
    15 threshold value Rso, Ri <Rso, said square Pi is preclassified as occupied and the laser telemetry system is required to check the distances on said square Pi.
    Figure 5 shows the procedure to preclassify a PA as free. He
    The procedure is executed if the movement of a possible vehicle (from the camera images) leaving the area of the square is detected. Thus, considering a place Pi, if near it the beginning of the movement of a possible vehicle with a value T> UT (exceeding a certain threshold UT) is detected or, in case there is no concealment, the recognition value of the soil Ri is greater than a threshold value Rsl, RI> Rsl, said preclassified
    25 Pi as free and the laser telemetry system is required to check the distances corresponding to said Pi square.
    Figure FlG.6 shows the procedure to confirm the change of status of a PA.
    For a pre-classified PA as occupied, this status is confirmed if it is satisfied that: I OPA -Dpc I <Uop and IOSA -Osc I> UOS. For a pre-classified PA as free, this status is confirmed if it is satisfied that: I OR PA -Dpc I> ULP or I OSA -Osc I <U LS 12
    5 MOOF OF EMBODIMENT The system of the present invention is composed of a video camera (1 1), an artificial vision processing module (1.2), a laser rangefinder (1 .5) with adjustable pan-till platform (1.4) , a module for controlling the orientation of the rangefinder (1 .3) and a module for merging video results and distances (1.7) to obtain the occupation of the monitored PAs. In the scheme of FIG. 1, the different chained elements are joined as their use is required.
    The fusion of the data obtained from the video images and the distance measurements obtained by the rangefinder provides a more reliable value of the occupancy status of the PAs. To obtain the information of the same PA from both sensors of the system, it is necessary to perform a calibration procedure during installation.
    Apart from the fact that the system must have a direct line of sight on the PAs to be supervised, there are no other restrictions regarding landmarks, beacons or any other invasive element in it. The area of occupation of the PAs must be configured a priori, but can be modified at will at any time since it does not imply infrastructure modification.
    Obtaining the occupation status of the PAs consists of the following 4 stages:
    one. Calibration of the adjustable rangefinder system and camera to work together.
    2. Processing through artificial vision: pre-classification of the change of state of each PA.
    3. Obtaining distance measurements by means of the laser rangefinder on the pre-classified APs.
    Four. Fusion of data obtained from the rangefinder and camera.
    Next, the operation of each of the previous blocks is detailed.
    5.1 Adjustable rangefinder and camera set calibration With this calibration it is desired to obtain a correspondence between points of the parking image with the points where distance measurements are made by the rangefinder, which implies knowing the pan-liIt orientation values to direct the rangefinder
    towards those points. The calibration procedure is performed in the assembly installation.
    This procedure consists of the following steps, described in Figure FIG.2:
    to. Definition of the PAs using a reflection prism that is being placed consecutively in the vertices of the floor of each square (manual placement of the prism). These locations are captured from the vision camera and from the laser rangefinder so that there are vertices that define each of the PAs.
    b. Camera Calibration - Step Performing the extrinsic calibration of the artificial vision camera with respect to the parking surface. Obtaining the homography matrix that relates the position of the PAs in the plane of the captured image. Obtaining the geometric transformation of the floor of a square with respect to the chamber from the measurements obtained by the rangefinder oriented towards the vertices of the same.
    C. Camera-Rangefinder calibration. Procedure for the geometric calibration of the adjustable rangefinder with respect to the camera. Once the actual position of the points defined for each square is known, the geometric transformation between the camera and the adjustable rangefinder is obtained.
    d. Obtaining a mapping of pan-tift orientations for the generation of measures at the desired points within the parking lot under analysis. Therefore, measurements are made at the central points of the squares at a height of zero meters (on the ground) and at a height equal to 1.5 meters aided by a reflection prism. The distance to the point on the floor of the PA can easily be occluded by other adjacent vehicles. By including the point at a height of 1.5 meters, the monitoring area for which the rangefinder has no occlusions is extended.
    and. Definition and generation of volumes 30 of each of the PAs to take into account the possible concealment between adjacent squares. A three-dimensional volume with a height of 1.5 meters in each square is considered to obtain which nearby squares are affected by a possible occlusion in case said square is occupied. 5.2 Artificial vision processing module
    The camera (FIG. 1 - block 1.1) provides a sequence of standard images on which the following artificial vision algorithms are executed (FIG. 1 - block 1.2).
    5.2.1 Vehicle detection algorithm An optical flow algorithm detects if there are any moving objects in the image sequence. In the areas of the image where there is no movement, a static background image is updated for use in the optical flow algorithm. Next, the algorithm detects which objects can be considered as vehicles since the area occupied by each detected object must be within threshold values depending on the location in the image (the vehicle with minimum / maximum dimensions is considered and get its size captured in the image using the current perspective projection of the camera with respect to the parking floor).
    5.2.2 Vehicle motion detection algorithm in a sequence of images
    A space · time filtering algorithm is applied based on [Babenko B., Yang M.H., and 8elongie S. "Visual tracking with online multiple instance learning". In Computer Vision and Pattern Recognition, 2009. CVPR 2009 IEEE Cont., Pages 983-990. 2009. https: /Idx.doi.org/10.1109/CVPR.2009.5206737J to obtain vehicle movement even in the presence of occlusions and detection errors in a sequence of images for a limited time. This algorithm provides a T value (in a range of values normalized between O and 1) that contains the result of the detection of the movement of a vehicle that either stops at a PA or exits a PA.
    5.2.3 Recognition of the floor of each parking space From the background image model obtained above, the recognition of the floor of each of the PAs is made from the information of its texture at different identifying points that may exist within of each square. If the occupation of a place by a vehicle or the total concealment of it is detected due to adjacent vehicles, the algorithm disables the recognition of the floor of that PA (not to repeatedly process that space unnecessarily). At the moment when the release of a square occupied by a vehicle is detected and there is no ground concealment by adjacent vehicles, reconnaissance of the square's floor is re-enabled. This algorithm provides an R value (in a range of values normalized between O and 1) that
    it contains the result of identification of the soil of said square based on its texture and characteristic points. 5.2.4 Obtaining pre-classified APs with change of status
    The execution of the previous vision algorithms, provide two output values (T and R) about the detection recognition.
    5.2.4.1 Preclassification of a place Pi as occupied Considering a place Pi, if near it stops the movement of a vehicle with a value T> UT (which exceeds a certain threshold UT) and the recognition value of its soil R¡ it is less than a threshold value Rso, R¡ <Rso, Pi is preclassified as occupied and the laser telemetry system is required to check the distances corresponding to said square Pi. The preclassification procedure of a PA as occupied is shown in Figure FIG. 4.
    5.2.4.2 Preclassification of a PI place as free Considering a place Pi, if near it the beginning of the movement of a possible vehicle with a value T> UT (exceeding a certain threshold UT) is detected or, if not there is concealment, the recognition value of the soil Ri is greater than a threshold value RsL, Ri> RsL, said P is preclassified as free and the telemetry system is required to be a check of the distances corresponding to said square Pi. The preclassification procedure of a PA as free is shown in Figure FIG. 5. 5.3 Obtaining distance measurements by means of the laser rangefinder on the PAs
    preclassified By incorporating an adjustable laser rangefinder, different distance measurements to previously generated points are obtained as points of the scene that help determine whether or not there is a vehicle in a PA (see figure FIG. 3). To conveniently orientate the laser rangefinder, a pan-tilt movement platform (with two degrees of freedom) is used. In FIG. 3, both angles are marked with rotation arrows: the horizontal pan angle and the vertical tilt angle.
    The detection procedure involves the control of the pan-tilt orientation platform to efficiently direct the rangefinder beam to the different PAs, where the vision system reports a possible change in occupancy
    The adjustable rangefinder is calibrated and there is a table with the distances measured from the sensor to the points of interest of each of the PAs. For each AP the corresponding pan-tilt control values necessary to orient the rangefinder and two are stored. Pre-calibrated distance measurements: Osc ~ measured distance to the central point on the floor of the square and Opt-measured distance from the central point of the square at a height of 1.5 m).
    5.4 Fusion of data obtained from the rangefinder and camera From the values provided by the artificial vision algorithm and the distances measured by the rangefinder oriented to the pre-classified APs, its interpretation is performed to generate a fusion result of both measurement systems obtaining an occupancy map of the supervised PAs and also generate a reliability value of the measure obtained. This procedure is shown in Figure FIG.6.
    5.4.1 Mark a PA as occupied
    From the information obtained on the pre-classified PAs with change of status to occupied it is confirmed that the state becomes occupied, if the current distance measured to the central point of the square at a height of 1.5 m (OPA) differs from the distance previously calibrated Opc less than a certain threshold value Uop (1 OPA ~ Opc I <Uop) and the current measured ground clearance of the square (OSA) differs from the previously calibrated distance Ose more than a margin determined by the Uos threshold (I OSA -Osc I> Uos). In another case, the change of state is not confirmed and the occupation of the place under analysis is not modified.
    The margins defined by the Uop and Uos thresholds are proportional to the distance to the Pi square under analysis.
    5.4.2 Mark a PA as free
    From the information obtained on the pre-classified places with change of state to free, it is confirmed that the status of a Pi square becomes free, if the current distance measured to the central point of the square at a height of 1.5 m (OPA) differs from the previously calibrated distance Opc more than a certain ULP threshold value (I OPA -Opc I> ULP) or, if there is a positive recognition of the ground (there is no concealment), that the current distance measured to the ground of the Square (OSA) differs from the previously calibrated distance Osc less than a threshold margin ULS (I OSA -Osc I <ULS). In another case, the change of state is not confirmed and the occupation of the place under analysis is not modified.
    The margins defined by the ULP and ULS thresholds are proportional to the distance to the Pi square under analysis.
    5.4.3 Reliability value of the occupancy status of each PA Each time the occupancy status of a PA is updated, a reliability value F of the occupancy status is provided by weighing the different values obtained in the vision and verification algorithms of the distance measurements with respect to the threshold values which are in the range of O to 1, 1 meaning greater reliability of the information regarding each of the parameters.
    Thus, as mentioned above, the process of detecting the entry / exit movement of a car to a PA provides a reliability value T in the range O to
    1. The soil recognition value R also provides a reliability value in the range O to 1.
    To obtain a reliability parameter in the comparison of distance measurements, the difference is normalized with respect to the threshold value considered (ground / 1.5m).
    In the following equations, the Mop Y Mas result values are obtained from comparing the distances measured at 1.5 m and on the ground, respectively, to check if a PA has been occupied:
    IDpA -Dpcl
    Mop = 1 -'-- '"" ;; --''-'-'
    UDP
    YOU
    Mos = 1; saturated to 1
    LOSA -Ose l
    In the following equations the MLP and MLS result values are obtained for comparing the distances measured at 1.5 m and on the ground, respectively, to check if a PA has been released:
    one ; saturated to 1
    Mas YMLP values are saturated to 1. Thus all Mxy parameters are in the range of
    Or to him.
    10 Finally, the reliability value Fx of the final state of occupation (X = O, Occupied or X = L, Free) of a PA is obtained by the following equation: Fx = alT + az R + a3 Mxs + a1 Mxp
    The sum of the different weights to the weighting of the parameters that come into play 15 for the calculation of the reliability F must be equal to 1.
    "" ~ al = 1
    ¿¡¡¡¡
    In this way, the reliability value of the occupation status F will also be given in the range Oa 1. 20
    The information on the occupation status of the PAs together with the measurement reliability value can be sent to other external systems or databases that confer an added value, according to the type of end user of the information provided by the system.
    权利要求:
    Claims (7)
    [1]
    1. System for detecting parking occupancy characterized by
    understands:5 a. A video camera.
    b. A module for processing artificial vision algorithms.
    C. A rangefinder orientation control module.
    d. An adjustable pan-tilt platform.
    and. A laser rangefinder 10 f. A pole height 1.5m,
    g. A prism of reflection.
    h. A module of differential measurement of distances.
    i. A module for merging data from the video camera and the
    rangefinder. 15 j. A communication module
    [2]
    2. System according to claim 1, characterized in that it comprises transmission means that send the occupancy information of the parking spaces (PAs) to other external systems or databases from the communication module.
    [3]
    3. Procedure for detecting parking occupancy. This claim consists of 4 stages that are listed and described below:
    [3]
     3.1 Calibration stage of the adjustable rangefinder system and camera:
    2S a. Definition of parking spaces. From the image captured by the camera of visual marks placed exprofeso in the vertices that define each parking space, these are delimited and their coordinates are recorded in a data table.
    b. Calibration Chamber-parking spaces. From delimiting vertices of
    30 each place an extrinsic calibration of the camera with respect to the parking floor is performed. This requires the help of a reflection prism. The prism is located in each of the delimiting vertices of the PAs and their position is captured in the camera image. All images captured and
    Prism positions are used for geometric calibration between the camera reference system and the PAs.
    C. Calibration of the laser rangefinder orientation system. The control values of the pan-tilt orientation of the rangefinder are stored at the central point of each square: a) on the ground and b) at a height of 1.5 meters. This requires the help of a reflection prism and a 1.5 nm post. Placed the prism in the ground, in the central point of each one of the PAs, the measures of distance and orientation between the rangefinder and these points are obtained. Placing the prism at a height of 1.5 m with the help of the pole located at the center point of each PA, the distance and orientation measurements between the rangefinder and these points are obtained. With the measurements made of orientation and distance to the points of the PAs, the geometric calibration of the orientation system of the laser rangefinder with respect to each of the PAs is obtained
    d. Calibration of the camera-rangefinder set. From the distances collected from the different points of the previous section and the camera calibration parameters, for each point of interest in the parking lot, a correspondence between the pan-tilt motion controls and xy coordinates of the pixel of the camera is obtained. image that captures that point. The information is stored in a correspondence table.
    and. Oeterminación delimited space volumes of each of the parking spaces and occlusions that generates adjacent spaces. A three-dimensional volume 1.5 meters high in each square is considered to obtain which nearby squares are affected by a possible occlusion in case said square is occupied, taking into account the camera's perspective on the ground.
    [3]
    3.2 Processing stage through artificial vision: pre-classification of the change of state of each parking space (PA):
    to. Application of an optical flow algorithm on the images captured from the camera. The static background image information is updated. From the image difference between the captured image and the background image information, moving objects are detected.
    b. Identification of possible objects as candidates for moving vehicles inside the parking lot by morphological characteristics of size depending on their location and perspective in the image.
    C. Identification of the direction of movement of a vehicle near a PA to be able to decide if it leaves free or occupies a seat depending on whether it was stopped and starts movement or if it is moving and stops. It is analyzed from an object tracking algorithm.
    d. Recognition of the soil of each PA from the background image / model, considering the texture and characteristic points of each square. It is enabled or disabled depending on the visibility of the soil according to the occupation of the PA under analysis and the adjacent squares.
    and. Preclassification of change of occupation status of a PA. From the identification measure (T) of the movement of a vehicle close to a PA and the recognition of the ground (R) it is decided to mark as possible change of status the occupation of a PA. From this information a list of pre-classified parking spaces with changes in the occupation status is obtained, which will be used to obtain distance measurements of the laser telemetry system.
    [3]
    3.3 Stage for obtaining distance measurements by means of the laser rangefinder on the pre-classified parking spaces (PAs):
    to. Orientation of a laser rangefinder towards the points of interest of pre-classified PAs through the use of a pan-tillo platform For this we rely on the list of pre-classified PAs and the correlation table obtained in the procedure described in step 3.2.
    b. Obtaining the measurements of current distances (current ground clearance -OSA and current distance to the post -OPA) of the pre-classified APs to be able to compare them with the stored values obtained in the calibration process (calibrated ground clearance -Osc and distance calibrated to the post -Opc). From the calibration described in step 3.1.c, for each PA two measurements are stored together with their corresponding pan-tilt control values necessary to orient the rangefinder and two distance measurements: the distance measurement (to the center point on the floor of the square and at a height of 1.5 m - obtained in stage 3.1.c).
    C. Definition of distance thresholds for the determination of occupancy of each PA. The decision on the status of occupation (free or occupied) of a PA will be made by comparing the actual measurements made at any time, between the chamber and the center of the square at heights of O and 1.5m, with the reference measures stored for each square to those same 2 points (center of the PA at heights of O and 1.5m). To confirm the status (free or busy), it is checked whether the differences between the measured and reference values are lower than the thresholds defined below. Therefore, four thresholds are defined for each place that delimit the measurement margins (two for occupied state and two for free state). In the case of occupied square (O) the thresholds are Uos - occupied floor threshold - and Uop - occupied threshold 1.5m high -: while in the case of free square (L) they are U lS - free ground floor-y U lP. -1,5m height free threshold. Once the system is installed and calibrated, the values of these thresholds remain fixed.
    [3]
     3.4 Stage of fusion of data obtained from the rangefinder and camera:
    to. Confirmation of the change of state of a parking space (PA) preclassified as occupied. The status of a pre-classified PA is confirmed as occupied if the current distance measured at the center point of the square at a height of 1.5 m (OPA) differs from the calibrated distance Opc less than a certain threshold value Uop (I OPA -Dpc I < Uop) and the current measured ground clearance of the square (OSA) differs from the previously calibrated distance Osc more than a margin determined by the Uos threshold (I OSA -Osc I:> UOS). In another case, the change of state is not confirmed and the previous occupation status is maintained (free place).
    b. Confirmation of the change of status of a pre-classified PA as free. The status of a pre-classified PA is confirmed as free if the current distance measured at the center point of the square at a height of 1.5 m (DPA) differs from the previously calibrated distance Dpc more than a certain ULP threshold value (I OPA -Opc I > ULP) or, if there is a positive recognition of the ground (there is no concealment), that the actual distance measured to the ground from the square (DSA) differs
    of the previously calibrated distance Dse less than a threshold margin ULS (I
    DSA -Dse I <ULS).Otherwise, the change of state is not confirmed and the status ofprevious occupation (occupied position).
    C. Obtaining a value of reliability of the occupation status for each PA. Each time the occupancy status of a PA is updated, a reliability value of the final occupancy status (Fo reliability value of a square marked as occupied, FI, reliability value of a square marked free) is provided. weighting the different values obtained in the vision algorithms with the weights at 'az, at 3, G4 for checking the distance measurements with respect to the threshold values, using the following equations:
    Po = at T + a2R + a3 MOS + a4Mop
    FI .. = a¡7 '+ a2R + a3MLS + a "MLP The weighting factors G¡ remain fixed after the installation of the system. T represents the measure of movement identification of a nearby vehicle and R the measurement of ground recognition The Mop and Mos values are the result of the comparison of the distances
    measures at 1.5 m -post and on the ground, respectively, when a square is marked as occupied:
    Mop = 1
    one; saturated to
    The MLP and MLS values are the result of the comparison of the distances measured at 1.5 m -post and on the ground, respectively, when a place is marked as free:
    MLP = I I 1; satl1rado to l
    -
    DpA Dpc
    类似技术:
    公开号 | 公开日 | 专利标题
    US9858816B2|2018-01-02|Determining parking space occupancy using a 3D representation
    US10127461B2|2018-11-13|Visual odometry for low illumination conditions using fixed light sources
    Wen et al.2015|Spatial-related traffic sign inspection for inventory purposes using mobile laser scanning data
    CN111491131B|2021-06-11|Method and apparatus for integrating object detection information detected by each object detector
    Zheng et al.2013|A practical roadside camera calibration method based on least squares optimization
    CN109084724A|2018-12-25|A kind of deep learning barrier distance measuring method based on binocular vision
    Batista et al.2014|Lane detection and estimation using perspective image
    Murray et al.2011|Mobile mapping system for the automated detection and analysis of road delineation
    You et al.2018|Joint 2-D–3-D traffic sign landmark data set for geo-localization using mobile laser scanning data
    CN105303580A|2016-02-03|Identification system and method of panoramic looking-around multi-camera calibration rod
    ES2680993B1|2019-03-27|System and procedure for detection of occupancy in car parks
    Bayerl et al.2015|Following Dirt Roads at Night Time: Sensors and Features for Lane Recognition and Tracking
    Bravo et al.2013|Outdoor vacant parking space detector for improving mobility in smart cities
    Tang2013|Development of a multiple-camera tracking system for accurate traffic performance measurements at intersections
    KR20210069385A|2021-06-11|Mapping device between image and space, and computer trogram that performs each step of the device
    Appiah et al.2011|Obstacle detection using stereo vision for self-driving cars
    Novais et al.2017|Community based repository for georeferenced traffic signs
    Rebut et al.2004|Road obstacles detection using a self-adaptive stereo vision sensor: a contribution to the ARCOS French project
    KR101209598B1|2012-12-07|Monitoring system
    Morris et al.2018|A Real-Time Truck Availability System for the State of Wisconsin
    CN111316324A|2020-06-19|Automatic driving simulation system, method, equipment and storage medium
    Nyaruhuma et al.2010|Line matching in oblique airborne images to support automatic verification of building outlines
    Sandidge2012|Truck height determination using digital video
    KR102373733B1|2022-03-15|Positioning system and method for operating a positioning system for a mobile unit
    JP2020077297A|2020-05-21|Position and posture estimation device
    同族专利:
    公开号 | 公开日
    ES2680993B1|2019-03-27|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US20130057686A1|2011-08-02|2013-03-07|Siemens Corporation|Crowd sourcing parking management using vehicles as mobile sensors|
    US20150339924A1|2014-05-21|2015-11-26|Douglas J. Cook|Parking space occupancy|
    法律状态:
    2018-09-11| BA2A| Patent application published|Ref document number: 2680993 Country of ref document: ES Kind code of ref document: A1 Effective date: 20180911 |
    2019-03-27| FG2A| Definitive protection|Ref document number: 2680993 Country of ref document: ES Kind code of ref document: B1 Effective date: 20190327 |
    优先权:
    申请号 | 申请日 | 专利标题
    ES201700190A|ES2680993B1|2017-03-10|2017-03-10|System and procedure for detection of occupancy in car parks|ES201700190A| ES2680993B1|2017-03-10|2017-03-10|System and procedure for detection of occupancy in car parks|
    [返回顶部]